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Lisa Computer User Testing Comments 


( Compiled by David Craig / 71533.606@compuserve.com ) 


° User Testing on the Cheap 
Bruce Tognazzini Apple Computer ca. 1989-90 p10 


This is how Larry Tesler, V.P. of Advanced Technology at 
Apple, carried out the extensive testing that resulted in the 
Lisa computer: 


Ive been personally involved in cheap user testing for about 
28 years, including my tests for the Lisa. The equipment was 
me, a Lisa, two chairs, a note pad and a pen. The subjects 
were right out of employee orientation. The tests ran one 
hour. I ran 3 to 5 subjects on each version of each 
application. I spent another few hours analyzing the notes 
and writing the report. Cheap. Effective. 


° An Interview with Wayne Rosing, Bruce Daniels, and Larry 
Tesler 


BYTE Magazine February 1983 


(Daniels) Another thing that weve done is user tests - taking 
our ideas and bringing in naive users and sitting them down 
and seeing what their impressions are. That has caused some 
changes, and I think thats all shown in the quality. 


(BYTE) Where did you get your naive users? 


(Tesler) Various places - the bulk of them were new Apple 
employees. We had a screening process. New Apple employees go 
through an orientation the first Monday morning theyre here. 
We handed out a questionnaire to the new employees about 
their previous experience with computers, word processors, 
video games, and that sort of thing, and then what kind of 
work they did. Someone in our training department screened 
all those vitae. Id go in and say I needed three user test 
subjects this week who have no word-processing experience but 
who are secretaries or accounting people to test out our 
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LisaCale. Shed go through and pick out some candidates and Id 
pick the ones I wanted, based on their experience for 
whatever test I was trying to run. We had about 50 tests this 
year in engineering to test out the software. 


(BYTE) The fact that you responded to the tests speaks well 
for the end product. 


(Tesler) We had a procedure for proposing changes, reviewing 
the changes, narrowing it down to a few choices, with certain 
criteria like consistency and parsimony. And then we actually 
implemented two or three of the various ways and tested them 
on users, and thats how we made the decisions. Sometimes we 
found that everybody was wrong. We had a couple of real 
beauties where the users couldnt use any of the versions that 
were given to them and they would immediately say, "Why dont 
you just do it this way?" and that was obviously the way to 
do it. So sometimes we got the ideas from our user tests, and 
as soon as we heard the idea we all thought, "Why didnt we 
think of that?" Then we did it that way. 


ro) Lisa Product Introduction Plan (PIP) 
Apple Computer January 1983 p43 


User Testing: These tests assure that the product is easy to 
use and that the manual, computer-assisted-instruction, and 
other instructional devices indeed help the user become 
skillful in the shortest time possible. POS Training supports 
the rest of POS in setting up and interpreting the results of 
user tests. 


° Lisa Questions and Answers 

Apple Computer February 1983 p17 

Q. How much time will I need to feel comfortable with Lisa? 
A. Our testing shows new users can learn to us the Lisa 
applications well enough to do useful work in about 1/2 hour. 
Mastering the applications, of course, depends on how 


frequently you use the Lisa, but it will be much faster than 
with conventional computers. 
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Q. You claim that you can learn Lisa applications in 1/2 
hour? How do you know? 


A. Apple has done extensive testing of both our software and 
our documentation. Weve found that the mean time to 
completing the "Getting Started" tutorial for each 
application was less than 30 minutes. These studies included 
a short test at the end of the tutorial to make sure that 
each user had learned enough about the application to do work 
on their own. Apple does not, however, claim that every user 
can accomplish this in 30 minutes. 


° The Past, Present, and Future of the Macintosh Desktop 
Dan Smith (Lisa Desktop Manager programmer) 1986 
Semaphore Signal 12 March 1986 


It took a fair amount of iterating to come up with something 
that appeared to be not only satisfying to the groups working 
there, but also something that went fairly well in our user 
testing, which we did quite a bit of. We tested mainly on 
people who had very little computer experience. We tried 
initially to make it mainly those who were in Apple, who had 
very limited experience, maybe new hires or people whose job 
didnt involve working directly with a computer. Further on 
into the project, we actually hired some people for testing 
purposes and we had a set of people in-house whose primary 
job was to gather testing feedback. We even set up a couple 
of testing rooms, and carefully recorded everything that 
happened, and tried to draw whatever conclusions we could. It 
took a fair amount of that to get us headed in the right 
direction. 


° Usability Testing in the Real World 

CHI86 Proceedings 

Bruce Tognazzini Apple Computer 1986 p213 

During the Lisa project, we did classical, exhaustive 
testing, often using new employees as subjects, occasionally 


winging off to Cincinnati to talk to officially "average" 
people. We over-focused, creating voluminous reports on "User 
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Reaction to Various Iconic Representations of Timepieces," 
while ignoring that it took 45 seconds for the user to drag 
the resulting Clock icon in from the disk. Users could walk 
to the next room and look at a real clock in less time. 
Fortunately, when it came time for the Macintosh, we were 
able to take the formal research of Lisa, filter it through 
Common Sense, and build a pragmatic, highly usable product. 


° The Mouse that Roars - A Day with Lisa 
George Taber 


FORTUNE Magazine, Apples Bid to Stay in the Big Time, 07 
February 1983, p38 


Is Lisa really as easy to use as Apple Computer has been 
vaunting? To find out, George M. Taber, an Apple // owner, 
recently spent a day testing the new machine for FORTUNE. His 
report: 


The Lisa is not as simple to use as a toaster or a telephone. 
But for anyone who has gone through the sometimes painful 
process of learning how to work a personal computer, Lisa is 
a dream. For those who have yet to touch a computer keyboard, 
Lisa will save weeks in the technological desert. If a person 
can get over the sticker shock of a $10,000 personal 
computer, he or she is likely to find Lisa both powerful and 
versatile. 


The key to Lisas charms is the mouse, the push-button control 
device that sits next to the computer and tells it what to 
do. The mouse eliminates clumsy computer commands that have 
to be typed into other personal machines. Thanks to the 
mouse, which takes about 15 minutes to master, executives can 
spend hours at a computer and never touch the keyboard. All 
instructions to the computer are given by pointing an arrow 
on the screen to a picture or word and then pushing the 
button on top of the mouse. The user moves the arrow on the 
screen by sliding the mouse across the desktop. If he wants 
to make a printed copy of the memo on the screen, for 
example, he moves the arrow to the word Print, which appears 
in the upper lefthand corner, and then presses the button. 
Presto. A copy is printed. Man has conquered machine. Anyone 
who can push a button can use Lisa. 
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Lisa's simplicity is clearly seen in a powerful tutorial 
program called the LisaGuide. Most personal computers still 
use torture-chamber instruction books that seem to be written 
by engineers for other engineers and that do everything 
except instruct. The introductory book for the IBM Personal 
Computer, for example, is so confusing that a guide to the 
guide has now been published. LisaGuide shows someone how to 
use the machine with the help of the computer itself. 
Instructions appear right on the screen in clear English. 
Then, just in case the lesson is still not understood, there 
is a demonstration complete with explanation and 
illustrations. In place of the words Syntax Error that appear 
on the screens of many other personal computers whenever 
something goes wrong, Lisa shows the picture of a Stop sign 
and then explains the problem. It takes about 45 minutes to 
do all the exercises in the LisaGuide. 


Lisa is above all a visual machine. The simulated desktop on 
the monitor screen presents a vivid picture of work in 
progress. When electronic folders are "picked up" with the 
help of the mouse, they suddenly zoom up into view, and other 
folders fade into the background. Lisa is perhaps most 
impressive turning numbers into graphs. With a few pushes on 
the mouses button, statistics are turned into a bar chart, a 
pie chart, or a graph. Executives who have to watch the 
progress of complex development projects through 
bureaucracies are likely to be infatuated with the flow 
charts that show a programs task, deadlines, and potential 
bottlenecks. The flow chart program may become one of Lisas 
most important innovations. 


While Lisa is powerful visually, she is less handy playing 
with words. The text processing program is the weakest aspect 
of the new computer. Anyone familiar with the office word 
processors or even those in some personal computers is likely 
to be disappointed by the slow and clumsy way Lisa erases 
sentences or moves words around a text. The keyboard is not 
as sensitive to the touch as an electric typewriter, and 
there is a brief but irritating delay between the time a 
letter is typed and its appearance on the screen. Apple 
engineers claim they will have corrected those problems 
before Lisa deliveries start this spring. 
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Apple's ambitious claim is that anyone can sit down at Lisa 
and be working on one of the six Lisa programs in about 20 
minutes. Like so many other promises of the computer age, 
that is unrealistic. It would take perhaps half a day of 
patient practice with Lisa and her mouse for someone with 
little or no experience to master the basics of the lady's 
tricks. By comparison, it would take at least a couple of 
weeks to reach the same level of proficiency on an Apple ][. 
Despite her failure to live up to that grand 20-minute 
promise, Lisa could claim as a motto the slogan on T-shirts 
worn by teenage girls: I may not be perfect, but parts of me 
are very good indeed. 


ro) Interesting misc. articles about user testing: 


User Testing - Why and How 

Tom Erickson 

Apple Computer 

Apple Direct magazine December 1989 


Conducting User Observations 
Kathleen Gomoll 

Apple Computer 

Apple Direct magazine January 1990 


Human Factors Testing in the Design of Xerox's 8010 "Star" 
Office Workstation 

William Bewley et al 

Xerox 

CHI83 Proceedings ACM 
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